Dialogue and Critical Thinking 1 DIALOGUE AS MEDIUM (AND MESSAGE) FOR TRAINING CRITICAL THINKING1 by

Standard internalist approaches to critical thinking insist that critical thinkers maintain conscious, deliberate access to the reasons for their beliefs and actions. A more useful approach is externalist, focusing on the reliability of different types of processes for generating beliefs and decisions under different circumstances. We describe an externalist approach to critical thinking based on dialogue. According to the theory, critical thinking is asking and answering questions about alternative possibilities in order to achieve some objective. Three perspectives are coordinated (by different individuals or inside a single head): a proponent, an opponent, and a referee. By asking and answering questions, the defender and challenger introduce new possibilities (mental models), understand them more completely, and learn one another’s beliefs and preferences. The referee, who represents an external perspective, regulates the dialogue so that it reliably achieves the participants’ objectives within the available time. "Critical Thinking through Dialogue" training takes trainees through four phases of a critical dialogue (identifying a disagreement, deciding how to resolve it, challenging and defending positions, and resolution), and presents principles associated with each phase. Tactical decision game scenarios were used prior to training, for practice, and for a post-training test. The training led participants to surface information not previously shared and to make effective use of it. They were more likely to recognize and deal with disagreements, to ask for and give reasons for positions, listen more carefully, and seek creative solutions rather than premature compromises. Current work is extending the theory and training to leadership skills. Dialogue and Critical Thinking 3 Approaches To Critical Thinking Definitions of critical thinking vary, but most share a common theme. Siegel (1997, p.14), for example, says that ...being a critical thinker requires basing one’s beliefs and actions on reasons... the beliefs and actions of the critical thinker, at least ideally, are justified by reasons for them which she has properly evaluated [italics in original]. Paul (1993) defines critical thinking as “a unique kind of purposeful thinking in which the thinker systematically and habitually imposes criteria and intellectual standards upon the thinking....” Implicit or explicit in many definitions (Johnson, 1996, chpt. 12) is that critical thinking involves the deliberate application of a proper evaluative criterion directly to beliefs and decisions and the reasons for arriving at them. These definitions (and many others; see Cohen, Salas, & Riedel, 2002, for a review) reflect a particular paradigm. Critical thinking has traditionally been conceptualized from an internalist point of view, which (as the name suggests) packs everything relevant to the evaluation of an intellectual product into the consciousness of an individual (Feldman & Conee, 2000; Plantinga, 1993a, pp. 3-29). From this perspective, it is inappropriate to credit a person for a correct judgment or decision if she cannot justify it by an explicit reason. If she cannot say why she judged or decided in the way she did, then she got it right by accident (P. Klein, 2000). Expert physicians who are unable to explain a diagnosis (Patel, Arocha, & Kaufman, 1999, p. 82) are irrational, or at the very least cannot be good critical thinkers. By the same token, there is a set of rules for proper reasoning that applies to all situations at all times; if adoption of a belief fits those rules, it cannot be faulted, no matter else is going on. For example, it is unfair to fault a judgment or decision by reference to information of Dialogue and Critical Thinking 4 which the person was unaware, even if readily available, or based on other rules, which are not intuitively compelling to that person. The purpose of this evaluation is “fairness” rather than effectiveness. As a consequence, critical thinking is supposed to evaluate a static, self-contained set of mental contents, in the light of universally compelling standards (e.g., logical, probabilistic, or decision theoretic) that specify how conclusions should be supported by reasons. Sosa (1991, p. 195) dubbed this view the “intellectualist model of justification.” A problem with internalism is that only a small subset of the reasons for a decision can ever be made explicit in critical thinking (Cherniak, 1986). As a result, the required absolute evaluative criteria do not exist. To varying degrees, everyone is in the same boat as the inarticulate expert physicians. Another problem is that, if applied “systematically and habitually” (e.g., as urged by Paul; also, Siegel, 1997: p. 16), the habit of asking for reasons leads to an infinite regress; to be a critical thinker is to be on a never-ending treadmill. Questions arise about the potential usefulness of training such skills for use in real-world domains like the Army tactical battlefield: Will critical thinking take too much time, undermine the will to fight, supplant experience and even expertise, stifle innovation, or disrupt coordination? Unfortunately, the current state of the field does not provide encouraging answers. Critical thinking textbooks tend to emphasize (i) basing judgments and decisions on formal or informal logic, probability theory, or decision theory, and (ii) the avoidance of common fallacies (Hansen & Pinto, 1995) and biases (e.g, Kahneman, Slovic, & Tversky, 1982). Critical thinking is still regarded implicitly as a form of inner purity. The critical thinker’s duty is to accept only beliefs seen clearly to follow from sound arguments, regardless of conditions or outcomes in the real world. There is little prescriptive research regarding pragmatic constraints on critical thinking that arise in time-sensitive domains. Dialogue and Critical Thinking 5 Our objective was to place critical thinking in a more realistic context for practitioners. To accomplish this, we sought a conceptualization that would: – Capture the idea of thinking about thinking without demanding that all (or even most) decision making be deliberative, or that all (or even most) reasons be made explicit. – Be usable in time-constrained situations. – Take account of constraints & obstacles due to social and organizational relationships. – Enhance the effectiveness of strategies actually used by proficient decision makers (e.g., recognition, story-building). – Be easy to teach, practice, and evaluate in real-world contexts. These objectives commit us to an externalist point of view (e.g., Papineau, 2003; Lipshitz & Cohen, 2003). From that perspective, evaluation is based on the reliability of the process that generated a judgment or a decision in real-world environments of the appropriate kind. Critical thinking has a different look and status in the externalist perspective: (i) Externalist evaluation is highly context-dependent, and the most effective processes or methods are often domain-specific rather than general. A judgment or decision is justified if it is generated by a process that reliably achieves objectives under relevant conditions (Goldman, 1992, chpt. 6). (ii) Reasons for a belief or decision will be spelled out to varying degrees depending on the context, and there will always be a large residual dependence on the reliability of relatively automatic perceptual and inferential processes (e.g., rapid recognition-primed decision making; Klein, 1993). (iii) Evaluation based on effectiveness (rather than “fairness”) is not limited to a person’s consciousness. Cognitive and social processes that expose views to outside challenge or actively seek information from the environment are likely to increase reliability (Goldman, 1992, chpt. 10) and may be part of critical thinking. (iv) Feasibility is built into the notion of reliability. Dialogue and Critical Thinking 6 Externalist criteria will favor strategies that are closely related to the way people already think over methods that are formally rigorous but impossible to implement (Lipshitz & Cohen, 2003). (v) No single method or process defines rationality. Critical thinking itself is not necessary for rationality, since intuitive or recognitional processes may be more reliable for achieving goals in familiar situations or when time is limited (Cohen, Freeman, & Wolf, 1996). At the other extreme, formal models might be worthwhile when their presuppositions are satisfied and an explicit computational implementation is available. Is there an externalist platform for critical thinking that realizes our objectives? Dialogue and Thought We propose an externalist perspective according to which critical thinking is, in essence, a form of dialogue. Dialogue theory studies reasoning and decision making as they actually occur in multiperson interactions rather than as a static set of logically related premises and conclusions (Hamblin, 1970; Rescher, 1977; van Eemeren & Grootendorst, 1983; van Eemeren et al., 1993). It seeks to identify the different types of argumentation (that is, the dynamic exchange of reasons for and against a conclusion) that are observed in conversation and the kinds of errors to which they are subject. Walton (1995, 1998) generalized the notion of dialogue beyond argumentation, defining it as any characteristic type of multiperson exchange with associated mutual expectations about roles, constraints, and purposes (Walton, 1998; Walton & Krabbe, 1995, p. 66). Based on that definition, he developed a classification of dialogue types, including negotiation, deliberation, inquiry, information seeking, and even quarreling. Dialogue theory blends descriptive and normative concerns, beginning in bottom up fashion with real-world conversations. Researchers start with observed types of interactive exchange and then build idealized models. These models constrain how each type of transaction should be Dialogue and Critical Thinking 7 conducted by participants who mutually recognize one another’s desire to cooperate to achieve the goal of that particular type of dialogue. Dialogues thus lend themselves to evaluation at two levels: First, how effective is the chosen type of dialogue for achieving the larger goals of the participants in the current context? Second, how effectively have participants conducted themselves to achieve the goals of that type of dialogue? Dialogue theory promises an evaluative framework that directly maps descriptive and cognitive analyses of actual exchanges onto prescriptive process constraints to identify where they diverge. Dialogue provides a paradigm for critical thinking that is descriptively richer and prescriptively more appropriate than internalist approaches based on logic, probability, or decision theory. Critical thinking dialogue leverages the functional similarity between rationally persuading another individual to accept or reject a position, and rationally determining for oneself whether a position is acceptable or not. A persuasion dialogue (Walton, 1998) or a critical discussion (von Eemeren & Grootendorst, 1992) externalizes necessary functions that must take place within an individual. In particular, both dialogue and critical thinking demand the adoption of different perspectives (e.g., a proponent, an opponent, and an external party who serves as facilitator, referee, or judge). Solo critical thinking may be fruitfully studied as a form of internal dialogue in which a single individual takes on these distinct dialectical roles (Walton, & Krabbe, 1995, p. 26). Moreover, variations among modes of critical thinking can be discriminated in terms of dialogue parameters, including purposes, roles, and constraints on types of questions and answers (including dialogues that are restricted to moves approved by logic and decision theory). Perhaps the most important functional similarity, and the one most highlighted by the dialogue perspective, is that both critical thinking and dialogues must be evaluated in terms of context-sensitive goals and costs. For example, in real-world argumentation Dialogue and Critical Thinking 8 (as contrasted with formal proof), there is no logically enforceable end to potential challenges and defenses. It follows that critical thinking cannot be evaluated as an abstract intellectual product (e.g., a fixed set of premises that “entail” a conclusion). Like a dialogue, it needs to be conceptualized and evaluated as a process. The similarity between critical thinking and a family of dialogue types may be based on more than functional analogy. First, a variety of developmental psychologists (e.g., Vygotsky, REF; Rogoff, REF; Tomasello, REF) have proposed that thought first develops as internalized speech and further, that we learn to reflect on and evaluate our own thoughts by responding to the thoughts and questions of others (Bogden, 2000). Second, as noted by Rieke and Sillars (1997), dialogue is the natural format for critical thinking by adults: ...research suggests that critical thinking is really a mini-debate that you carry on with yourself. What is often mistaken for private thought is more likely an “internalized conversation (Mead REF), an “internal dialogue” (Mukarovsky REF), or an “imagined interaction” (Gotcher and Honeycutt REF). Third, there is an even more direct reason for a dialogue-based theory of critical thinking. Thinking skills are not only learned in social interaction but continue to be manifested in social contexts (Hutchins, 1995). Much critical thinking takes place in a team or group context, in which dialogue plays a literal role in decision making. Dialogues are the interactions by means of which members of a team or group pool information and insights to solve a problem, resolve competing goals, build up shared understanding of the situation and task, and over time construct relationships that improve team cohesiveness and trust (Cohen, 2004). The fastest road to improved critical thinking in both an individual and a team may be training for critical dialogue. Dialogue and Critical Thinking 9 A Theory Of Critical Thinking As Dialogue As shown in Figure 1, our definition of critical thinking has three parts. Critical thinking is (1) a question and answer (or more specifically, a challenge and defend) dialogue with oneself or others (2) about alternative possibilities (3) carried out for a purpose. The dialogue-based theory of critical thinking draws on and synthesizes research on three separate topics: (1) descriptive and prescriptive models of critical discussion in which a proponent must defend a claim against an opponent or critic, typically by asking and answering questions (Walton, 1998; von Eemeren & Grootendorst, 1992); (2) cognitive theories of reasoning according to which alternative possible situations are represented by mental models (Johnson-Laird, 1983; JohnsonLaird & Byrne, 1991; Johnson-Laird et al., 1999); and (3) cognitive research that helps us assess the reliability of the processes by means of which we form beliefs and make choices – either by statistical measurement of outcomes (Hammond, 1993) or by research on expert performance (Orasanu & Connolly, 1993; Cohen et al. 1996). Mental models are alternative “realities” regarded as possible by the parties to the discussion, circumscribed to include only variables currently regarded as relevant. A particular mental model may include not only beliefs about the situation and the significance of evidence, but also preferences, goals, and intentions to act. For the purposes of critical thinking, a decision maker’s current situation understanding and plan is the set of mental models that are possible, or under consideration, in his or her present mental state. This set captures not only what the decision maker firmly believes or intends, but also the beliefs and intents that she regards as relevant but uncertain (i.e., whose truth values vary across alternative models). A critical dialogue coordinates three perspectives or roles that reflect opposing attitudes toward uncertainty regarding an issue. Occupants of the first role (the proponent) try to reduce Dialogue and Critical Thinking 10 uncertainty, by eliminating mental models in which the defended position is false. They present a position (an initial mental model) and defend it against challenges by adding further elements (i.e., reasons) that are consistent only with the truth of the position. Occupants of the second role (the opponent) try to increase doubt, by expanding the number of mental models to include some in which the defended position is false. They either demand reasons where none are present or introduce further elements (i.e., rebuttals) that eliminate existing reasons or neutralize the constraints that those reasons impose on the conclusion. In a somewhat more complex, symmetrical dialogue, both proponent and opponent also play the other’s role with respect to a second, competing position. Thus, as questions are asked and answered, critical dialogue alters both the number of mental models under consideration and the number of variables used to characterize them, over a series of “moves” that are parts of complementary opposing strategies. This process of questioning and defending mental models is adopted because of its reliability for achieving the purposes of the participants within the available time. Occupants of the third role (the referee or facilitator) regulate the process at two levels: the internal relevance of moves by each player to the goals of the dialogue and the external contribution of the dialogue as a whole to achieving a larger task or purpose within the available time. Despite their opposing roles within the dialogue, the proponent and opponent cooperate to accomplish a shared overall purpose, e.g., to test the acceptability of an uncertain position (which may include both assessment and plan) which affects the accomplishment of a larger task or activity. As a byproduct, because each may introduce factors not considered by the other, they increase their understanding of the situation along with their understanding of the other’s point of view. Critical dialogues, when successful, should therefore improve decisions and situation understanding (by bringing more information to bear) and improve mutual or shared knowledge Dialogue and Critical Thinking 11 (by exposing who knows or believes what). Each of the three components of critical thinking shown in Figure 1 is associated with distinctive metrics of success, which progress from internal to external in their focus: 1. At its innermost core critical thinking involves selective consideration of alternative possible states of affairs, i.e., ensembles of mental models introduced by the proponent and opponent, respectively. Metrics of performance at this level include both external criteria (typically focused on individual issues) and internal criteria (focused on relationships among positions on different issues). Each of these has two variants: one concerned with maximizing (or, for opponents, minimizing) certainty, and the other with accurately representing the uncertainty that remains. Thus, external metrics include: (i) the amount of correspondence between individual claims and current observations or substantiated empirical generalizations, and (ii) appropriate representation of the uncertainty of a position by the relative number of possibilities in which it is false. Internal metrics include: (iii) coherence among the claims that co-exist within any given model, both among themselves and with prior knowledge, and (iv) appropriate representation of the overall plausibility of a mental model by the extent to which it is represented in the ensemble of possible worlds. (i) and (ii) correspond to resolution and calibration, which are quantitative measures from frequentist probability theory (REF). Coherence and plausibility as referred to in (iii) and (iv) measure the degree to which a given mental model is a good theory. They may be based on explanatory principles and criteria within a relevant domain of inquiry, and therefore draw heavily on implicit, domain-specific standards of when it makes sense to introduce new explanatory variables to a model. General ideas of what constitutes a good theory, which trade off the Dialogue and Critical Thinking 12 number of parameters against predictive utility, have been developed by Sober & XXX (REF) and others. 2. Mental models are embedded within a process of critical questioning which motivates the generation, elaboration, and evaluation of possibilities. As noted, such dialogues may take place within a single individual, or they may be conducted among different individuals. Critical questioning is evaluated by reference to norms for conducting the appropriate kinds of critical dialogue, that is, constraints or rules that reliably enable the purpose of the dialogue to be efficiently accomplished. Such rules may be designed (and evaluated in terms of) specific purposes, for example, to keep the conversation moving until there is a small enough chance that relevant information has not yet been brought forward. For example, rules may require the proponent on any given turn to either respond to a challenge by the opponent or concede, and may require the opponent on any given turn to either raise a challenge or drop her opposition. 3. At the outermost layer, critical thinking is a judgment about the reliability of a cognitive strategy, trait, or social process for achieving goals under various conditions. Mental models may be cogent and the challenges and defenses relevant, but if the dialogue itself will not contribute to important goals within the time available, it should not occur. Different types of critical dialogue, as well as other processes like pattern recognition or the nominal group technique, are among the available cognitive or social tools that might be utilized to effectively generate or modify beliefs and decisions. Moreover, types of critical dialogues can be differentiated by the depth of probing to which a proponent must respond and the scope of the possibilities that may be considered, depending, for Dialogue and Critical Thinking 13 example, on whether the dialogue is intended to solve a time sensitive pragmatic issue or dig out and evaluate fundamental but hidden assumptions. In sum, critical thinking skill is exemplified by the interaction of challenges that introduce alternative possibilities and defenses that eliminate alternative possibilities – with the proximal goal of testing the acceptability of a claim and the distal goal of reliably contributing to the accomplishment of a larger task. More detailed discussion of this theory can be found in Cohen (2000) and Cohen, Salas, & Riedel (2002). An extension and generalization of it can be found in Cohen (2004). Hypotheses. The common knowledge effect is a well-documented shortcoming of group decision making that critical dialogue might mitigate. Group members are more likely to discuss information they already hold in common, even when there is more valuable unshared information; moreover, even when unshared information is mentioned, it has less impact on group decisions (Stasser, 1996; 1999). We predicted that teams trained in critical dialogue would be more likely to (i) pool information and (ii) use the pooled information to develop novel solutions. More specifically, training would increase the frequency with which team members discussed information not already held in common by members of the team and the frequency with which they made effective use of that information. Training Critical Thinking Through Dialogue Based on these ideas, we developed a training package called Critical Thinking through Dialogue. At the time of the study reported here, the training package was not stand alone, but was presented via slides and handouts by an instructor. Classes generally consisted of a group of 2 to 4 officers, usually of the same rank. Tactical decision games from the Marine Corps Gazette were used for testing before and after training, as well as for demonstration and practice during Dialogue and Critical Thinking 14 training (Schmitt, 1994). These are one-page descriptions of tactical situations, in which readers adopt the perspective of a company or battalion commander and then develop brief operations orders for their subordinates. (See below for more details.) The training began with a discussion of the concept of a critical dialogue, as a type of “game” in which the skills of critical thinking could be learned and put into practice. The instructor provided an overview of the three roles (proponent, opponent, and referee / facilitator) and associated rules. The instructor emphasized that the roles in critical dialogue need not correspond in any specific way to rank. Within the context of this game, the commander is voluntarily subject to the same rules as other players. The instructor then explained the four phases of a critical thinking dialogue (adapted from von Eemeren & Grootendorst, 1992): (1) Identifying a disagreement or uncertainty, (2) deciding how to resolve it, (3) challenging, defending, and modifying positions, and (4) resolution of the problem (Table 1). The presentation of each phase was accompanied by discussion of tasks and principles associated with each phase, guided practice, and feedback during and after the practice. Participants were taught rules specific to each phase as well as more general principles for critical dialogue. For many of these, they were shown common ways in which it tends to be violated (called “fouls”), and examples of each kind of violation. Two of the most important rules are shown in Table 2. Finally, the role of the referee / facilitator was explained: • Prioritize use of time. Direct discussion to priority issues that need to be decided soonest. Stop the discussion of an issue whenever an immediate decision is necessary on that issue. Stop the discussion when continuation is not likely to be fruitful or when other high-priority issues need to be discussed. Dialogue and Critical Thinking 15 • Keep discussion going if someone appears to concede soon. (Example: We still have some time. Don’t give up yet. Work harder to come up with a better defense of your position or a modification that meets the objections.) • Energize discussion if it gets into a rut or peters out. (Example: Don’t repeat the same points. Come up with new ideas. An infallible crystal ball says there are other problems with the plan. What are they?) • Call foul if a party violates the rules (e.g., Aren’t you changing the subject? Let’s stay on this topic for now.)